Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants

Dust-wind interactions can intensify aerosol pollution over eastern China.

Eastern China has experienced severe and persistent winter haze episodes in recent years due to intensification of aerosol pollution. In addition to anthropogenic emissions, the winter aerosol pollution over eastern China is associated with unusual meteorological conditions, including weaker wind speeds. Here we show, based on model simulations, that during years with decreased wind speed, large decreases in dust emissions (29%) moderate the wintertime land-sea surface air temperature difference and further decrease winds by -0.06 (±0.05) m s-1 averaged over eastern China. The dust-induced lower winds enhance stagnation of air and account for about 13% of increasing aerosol concentrations over eastern China. Although recent increases in anthropogenic emissions are the main factor causing haze over eastern China, we conclude that natural emissions also exert a significant influence on the increases in wintertime aerosol concentrations, with important implications that need to be taken into account by air quality studies

GIS-based Economic Cost Estimation of Traffic Accidents in St. Louis, Missouri

AbstractThe economic loss due to total traffic accidents in St. Louis remains high every year. This paper presents an effective approach to spatially identifying potential casualty areas and their economic losses. In this study, five years of traffic accident data, from 2007 to 2011, collected in the City of St. Louis and the adjacent counties, is used. Using Geographic Information System (GIS)-based techniques, e.g. Kernel Density Estimation (KDE), two maps are generated and compared: 1) traffic accident rate map based on the number of traffic accidents per year and 2) the economic costs map. The locations with high economic costs but with low accident rates are identified and shown in a 3-D visualization format. The results can be used as a foundation for the traffic accident cost estimation related research and serves as a guideline for practitioners to investigate the areas with high traffic accident severity levels

Identifying the Effects of Social Media on Health Behavior: Data from a Large-Scale Online Experiment

Sedentary lifestyle is an escalating epidemic. Little is known about whether or how social media can be used to design a cost-effective solution for sedentary lifestyle. In this article we describe the data from a randomized controlled trial (RCT) that evaluated two prominent strategies for conducting exercise interventions using elements of social media: motivational media campaigns and online peer networks. The data file includes 217 participants’ basic demographic information, number of exercise class enrollments over 13 weeks, and self-reported number of days for exercise activities in the previous 7 days at baseline. Among the 217, 164 also have data on self-reported number of days for exercise activities at the post-program. Data are supplied with this article. The interpretation of these data can be found in the research article published by the authors in Preventive Medicine Reports in 2015[1]

Efficacy and Causal Mechanism of an Online Social Media Intervention to Increase Physical Activity: Results of a Randomized Controlled Trial

Objective: To identify what features of social media – promotional messaging or peer networks – can increase physical activity. Method: A 13-week social media-based exercise program was conducted at a large Northeastern university in Philadelphia, PA. In a randomized controlled trial, 217 graduate students from the University were randomized to three conditions: a control condition with a basic online program for enrolling in weekly exercise classes led by instructors of the University for 13 weeks, a media condition that supplemented the basic program with weekly online promotional media messages that encourage physical activity, and a social condition that replaced the media content with an online network of four to six anonymous peers composed of other participants of the program, in which each participant was able to see their peers\u27 progress in enrolling in classes. The primary outcome was the number of enrollments in exercise classes, and the secondary outcomes were self-reported physical activities. Data were collected in 2014. Results: Participants enrolled in 5.5 classes on average. Compared with enrollment in the control condition (mean = 4.5), promotional messages moderately increased enrollment (mean = 5.7, p = 0.08), while anonymous social networks significantly increased enrollment (mean = 6.3, p = 0.02). By the end of the program, participants in the social condition reported exercising moderately for an additional 1.6 days each week compared with the baseline, which was significantly more than an additional 0.8 days in the control condition. Conclusion: Social influence from anonymous online peers was more successful than promotional messages for improving physical activity. Clinical Trial Registration: ClinicalTrials.gov: NCT02267369